Ensemble of deep learning language models to support the creation of living systematic reviews for the COVID-19 literature: a retrospective study (preprint)

Background: The COVID-19 pandemic has led to an unprecedented amount of scientific publications, growing at a pace never seen before. Multiple living systematic reviews have been developed to assist professionals with up-to-date and trustworthy health information, but it is increasingly challenging for systematic reviewers to keep up with the evidence in electronic databases. We aimed to investigate deep learning-based machine learning algorithms to classify COVID-19 related publications to help scale-up the epidemiological curation process. Methods: In this retrospective study, five different pre-trained deep learning-based language models were fine-tuned on a dataset of 6,365 publications manually classified into two classes, three subclasses and 22 sub-subclasses relevant for epidemiological triage purposes. In a k-fold cross-validation setting, each standalone model was assessed on a classification task and compared against an ensemble, which takes the standalone model predictions as input and uses different strategies to infer the optimal article class. A ranking task was also considered, in which the model outputs a ranked list of sub-subclasses associated with the article. Results: The ensemble model significantly outperformed the standalone classifiers, achieving a F1-score of 89.2 at the class level of the classification task. The difference between the standalone and ensemble models increases at the sub-subclass level, where the ensemble reaches a micro F1-score of 70% against 67% for the best performing standalone model. For the ranking task, the ensemble obtained the highest recall@3, with a performance of 89%. Using an unanimity voting rule, the ensemble can provide predictions with higher confidence on a subset of the data, achieving detection of original papers with a F1-score up to 97% on a subset of 80% of the collection instead of 93% on the whole dataset. Conclusion: This study shows the potential of using deep learning language models to perform triage of COVID-19 references efficiently and support epidemiological curation and review. The ensemble consistently and significantly outperforms any standalone model. Fine-tuning the voting strategy thresholds is an interesting alternative to annotate a subset with higher predictive confidence.

How to update a living systematic review and keep it alive during a pandemic: a practical guide (preprint)

Background The covid-19 pandemic has highlighted the role of living systematic reviews. The speed of evidence generated during the covid-19 pandemic accentuated the challenges of managing high volumes of research literature.Methods In this article, we summarise the characteristics of ongoing living systematic reviews on covid-19 and we follow a life cycle approach to describe key steps in a living systematic review.Results We identified 97 living systematic reviews on covid-19, which focused mostly on the effects of pharmacological interventions (n = 46, 47%) or the prevalence of associated conditions or risk factors (n = 30, 31%). The scopes of several reviews overlapped considerably. Most living systematic reviews included both observational and randomised study designs (n = 45, 46%). Only one third of the reviews has been updated at least once (n = 34, 35%). We address practical aspects of living systematic reviews including how to judge whether to start a living systematic review, methods for study identification and selection, data extraction and evaluation, and give recommendations at each step, drawing from our own experience. We also discuss when it is time to stop and how to publish updates.Conclusions Methods to improve the efficiency of searching, study selection, and data extraction using machine learning technologies are being developed, their performance and applicability, particularly for reviews based on observational study designs should improve, and ways of publishing living systematic reviews and their updates will continue to evolve. Finally, knowing when to end a living systematic review is as important as knowing when to start.

Occurrence and transmission potential of asymptomatic and presymptomatic SARS-CoV-2 infections: update of a living systematic review and meta-analysis (preprint)

ABSTRACT BACKGROUND Debate about the level of asymptomatic severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection continues. The amount of evidence is increasing and study designs have changed over time. We updated a living systematic review to address three questions: (1) Amongst people who become infected with SARS-CoV-2, what proportion does not experience symptoms at all during their infection? (2) What is the infectiousness of asymptomatic and presymptomatic, compared with symptomatic, SARS-CoV-2 infection? (3) What proportion of SARS-CoV-2 transmission in a population is accounted for by people who are asymptomatic or presymptomatic? METHODS AND FINDINGS The protocol was first published on 1 April 2020 and last updated on 18 June 2021. We searched PubMed, Embase, bioRxiv and medRxiv, aggregated in a database of SARS-CoV-2 literature, most recently on 6 July 2021. Studies of people with PCR-diagnosed SARS-CoV-2, which documented symptom status at the beginning and end of follow-up, or mathematical modelling studies were included. Studies restricted to people already diagnosed, of single individuals or families, or without sufficient follow-up were excluded. One reviewer extracted data and a second verified the extraction, with disagreement resolved by discussion or a third reviewer. Risk of bias in empirical studies was assessed with a bespoke checklist and modelling studies with a published checklist. All data syntheses were done using random effects models. Review question (1): We included 130 studies. Heterogeneity was high so we did not estimate a mean proportion of asymptomatic infections overall (interquartile range 14-50%, prediction interval 2-90%), or in 84 studies based on screening of defined populations (interquartile range 20-65%, prediction interval 4-94%). In 46 studies based on contact or outbreak investigations, the summary proportion asymptomatic was 19% (95% CI 15-25%, prediction interval 2-70%). (2) The secondary attack rate in contacts of people with asymptomatic infection compared with symptomatic infection was 0.32 (95% CI 0.16-0.64, prediction interval 0.11-0-95, 8 studies). (3) In 13 modelling studies fit to data, the proportion of all SARS-CoV-2 transmission from presymptomatic individuals was higher than from asymptomatic individuals. Limitations of the evidence include high heterogeneity and high risks of selection and information bias in studies that were not designed to measure persistently asymptomatic infection, and limited information about variants of concern or in people who have been vaccinated. CONCLUSIONS Based on studies published up to July 2021, most SARS-CoV-2 infections were not persistently asymptomatic and asymptomatic infections were less infectious than symptomatic infections. Summary estimates from meta-analysis may be misleading when variability between studies is extreme and prediction intervals should be presented. Future studies should determine the asymptomatic proportion of SARS-CoV-2 infections caused by variants of concern and in people with immunity following vaccination or previous infection. Without prospective longitudinal studies with methods that minimise selection and measurement biases, further updates with the study types included in this living systematic review are unlikely to be able to provide a reliable summary estimate of the proportion of asymptomatic infections caused by SARS-CoV-2. REVIEW PROTOCOL Open Science Framework ( https://osf.io/9ewys/ ) AUTHOR SUMMARY Why was this study done? ▪ The proportion of people who will remain asymptomatic throughout the course of infection with severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), the cause of coronavirus disease 2019 (covid-19), is debated. ▪ Studies that assess people at just one time point overestimate the proportion of true asymptomatic infection because those who go on to develop covid-19 symptoms will be wrongly classified as asymptomatic, but other types of study might underestimate the proportion if, for example, people with symptoms are more likely to be included in a study population. ▪ The number of published studies about SARS-CoV-2 is increasing continuously, types of studies are changing and, since 2021, vaccines have become available, and variants of concern have emerged. What did the researchers do and find? ▪ We updated a living systematic review through 6 July 2021, using automated workflows that speed up the review processes, and allow the review to be updated when relevant new evidence becomes available. ▪ In 130 studies, we found an interquartile range of 14-50% (prediction interval 2-90%) of people with SARS-CoV-2 infection that was persistently asymptomatic; owing to heterogeneity, we did not estimate a summary proportion. ▪ Contacts of people with asymptomatic SARS-CoV-2 infection are less likely to become infected than contacts of people with symptomatic infection (risk ratio 0.38, 95% CI 0.16-0.64, prediction interval 0.11-0.95, 8 studies). What do these findings mean? ▪ Up to mid-2021, most people with SARS-CoV-2 were not persistently asymptomatic and asymptomatic infection was less infectious than symptomatic infection. ▪ In the presence of high between-study variability, summary estimates from meta-analysis may be misleading and prediction intervals should be presented. ▪ Future studies about asymptomatic SARS-CoV-2 infections caused by variants of concern and in people with immunity following vaccination or previous infection should be specifically designed, using methods to minimise biases in the selection of study participants and in ascertainment, classification and follow-up of symptom status.

Practices and Behaviors During the COVID-19 Pandemic in Patients With Rheumatoid Arthritis Who Attended Previously to An Educational Program. A Cross-Sectional Study (preprint)

Background: To describe the practices and behaviors of patients with rheumatoid arthritis (RA) who attend to a face-to-face education program, during the quarantine of the COVID-19 pandemic. Methods: Patients who attended previously a face-to-face education program, responded to a telephonic survey in July 2020. The survey included questions about their practices related to the COVID-19 pandemic, SARS-Cov-2 symptoms, adherence to rheumatoid arthritis treatment, virtual rheumatology consultancy compliance and, the influence of news on their adherence. Results: A total of 260 patients participated in a survey. In July 2020 88% of patients had accessed a telemedicine-based and 12% a face-to-face rheumatology consultation. 3.5% of patients reported having been less adherent to pharmacological therapy due to information received through media or social networks. In general patients had been compliant with COVID-19 prevention recommendations.  Only one patient was positive for SARS-CoV-2 and reported only flu symptoms without any complications. Patients highlighted the necessity to have information and education about the relationship between rheumatoid arthritis, its treatment, and COVID-19. Conclusions: An educational program is a helpful tool to maintain high adherence rates to the RA treatment despite of the new challenges associated to the pandemic; Patient-centered education programs should continue to address the patient's concerns and beliefs about their disease and COVID-19.

Outbreaks of publications about emerging infectious diseases: the case of SARS-CoV-2 and Zika virus (preprint)

BackgroundOutbreaks of infectious diseases generate outbreaks of scientific evidence. In 2016 epidemics of Zika virus emerged, largely in Latin America and the Caribbean. In 2020, a novel severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) caused a pandemic of coronavirus disease 2019 (COVID-19). We compared patterns of scientific publications for the two infections over time. MethodsWe used living systematic review methods to search for and annotate publications according to study design. For Zika virus, a review team performed the tasks for publications in 2016. For SARS-CoV-2, a crowd of 25 volunteer scientists performed the tasks for publications up to May 24, 2020. We used descriptive statistics to categorise and compare study designs over time. FindingsWe found 2,286 publications about Zika virus in 2016 and 21,990 about SARS-CoV-2 up to 24 May 2020, of which we analysed a random sample of 5294. For both infections, there were more epidemiological than laboratory science studies. Amongst epidemiological studies for both infections, case reports, case series and cross-sectional studies emerged first, cohort and case-control studies were published later. Trials were the last to emerge. Mathematical modelling studies were more common in SARS-CoV-2 research. The number of preprints was much higher for SARS-CoV-2 than for Zika virus. InterpretationSimilarities in the overall pattern of publications might be generalizable, whereas differences are compatible with differences in the characteristics of a disease. Understanding how evidence accumulates during disease outbreaks helps us understand which types of public health questions we can answer and when. FundingMJC and HI are funded by the Swiss National Science Foundation (SNF grant number 176233). NL acknowledges funding from the European Unions Horizon 2020 research and innovation programme - project EpiPose (grant agreement number 101003688). DBG is funded by the Swiss government excellence scholarship (2019.0774) and the Swiss School of Public Health Global P3HS.

FALSE-NEGATIVE RESULTS OF INITIAL RT-PCR ASSAYS FOR COVID-19: A SYSTEMATIC REVIEW (preprint)

BackgroundA false-negative case of severe acute respiratory syndrome coronavirus 2 (SARS-CoV- 2) infection is defined as a person with suspected infection and an initial negative result by reverse transcription-polymerase chain reaction (RT-PCR) test, with a positive result on a subsequent test. False-negative cases have important implications for isolation and risk of transmission of infected people and for the management of coronavirus disease 2019 (COVID-19). We aimed to review and critically appraise evidence about the rate of RT-PCR false-negatives at initial testing for COVID-19. MethodsWe searched MEDLINE, EMBASE, LILACS, as well as COVID-19 repositories including the EPPI-Centre living systematic map of evidence about COVID-19 and the Coronavirus Open Access Project living evidence database. Two authors independently screened and selected studies according to the eligibility criteria and collected data from the included studies. The risk of bias was assessed using the Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) tool. We calculated the proportion of false-negative test results with the corresponding 95% CI using a multilevel mixed-effect logistic regression model. The certainty of the evidence about false- negative cases was rated using the GRADE approach for tests and strategies. All information in this article is current up to July 17, 2020. ResultsWe included 34 studies enrolling 12,057 COVID-19 confirmed cases. All studies were affected by several risks of bias and applicability concerns. The pooled estimate of false-negative proportion was highly affected by unexplained heterogeneity (tau-squared= 1.39; 90% prediction interval from 0.02 to 0.54). The certainty of the evidence was judged as very low, due to the risk of bias, indirectness, and inconsistency issues. ConclusionsThere is a substantial and largely unexplained heterogeneity in the proportion of false-negative RT-PCR results. The collected evidence has several limitations, including risk of bias issues, high heterogeneity, and concerns about its applicability. Nonetheless, our findings reinforce the need for repeated testing in patients with suspicion of SARS-CoV-2 infection given that up to 54% of COVID-19 patients may have an initial false-negative RT-PCR (certainty of evidence: very low). An update of this review when additional studies become available is warranted. Systematic review registrationProtocol available on the OSF website: https://osf.io/gp38w/